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DETAILED ACTION 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 20 May 
2010 has been entered. 

Response to Amendment 

2. The amendment filed on 20 May 2010 has been entered. Claims 1,21, 24, 26, 
30-33, 40, 42-45, 48, 53, 55, 58, 60, 61 , 64-66 have been amended. Claims 46 and 47 
have been canceled. Claims 1 , 4-7, 9, 10, 12-17, 19-33, 36, 40, 42-45 and 48-66 are 
now pending. 

Response to Arguments 

3. Applicants' arguments with respect to claims 1 , 33, 40 and 66 have been 
considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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5. Claims 1, 4-7, 9, 10, 12-17, 19-31, 33, 36, 40, 42-45 and 48-66 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Zwaneveld (CA 2129925) in 
view of Hale (US 6,785,539) and further in view of Miyaoku et al (US 2003/0051252; 
referred hereinafter as 'Miyaoku'). 

Regarding claim 1 , Zwaneveld teaches a captioning system for providing 
captions for a presentation to a user, the captioning system comprising: 
a caption store (storage element 1 1 ) configured to store one or more sets of captions 
each set being associated with one or more presentations and each set comprising a 
plurality of captions for playout at different timings during the associated presentation 
(page 9, line 1 8 - page 1 0, line 21 ; for display cue stamps comprises a display time 
stamp, each display time stamp comprising a first time and second time respectively 
marking the beginning and the end of the display of a subtitle). 

However, Zwaneveld does not explicitly teach: 
a portable user device having: 

a first receiver configured to receive, from said caption store, at least one set of captions 
for storage in the user device or to receive a sequence of captions for a presentation to 
be made to a user associated with the user device; 

a microphone configured to receive a wireless acoustic signal that is time synchronized 
with the presentation for use in defining the timing during the presentation at which each 
caption is to be output to the user associated with the user device; 
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a caption output circuit configured to output each received caption to the user 
associated with the user device; and 

a tinning controller configured to process said received wireless acoustic signal to 
determine the timing during the presentation at which each caption should be output 
and configured to control said caption output circuit so that said each caption is output 
to said user at the determined timing. 

In analogous art, Hale teaches: 

a portable user device (portable device 14) having: 

a first receiver configured to receive, from said caption store, at least one set of captions 
for storage in the user device or to receive a sequence of captions for a presentation to 
be made to a user associated with the user device (col. 2, line 63 - col. 3, Iine3; col. 3, 
lines 17-21 - for the portable user device receives caption texts and stores them); 

a caption output circuit configured to output each received caption to the user 
associated with the user device; and a timing controller configured to process said 
received wireless [acoustic] signal to determine the timing during the presentation at 
which each caption should be output and configured to control said caption output 
circuit so that said each caption is output to said user at the determined timing (col. 4, 
line 61 - col. 5, linel 5; col. 5, lines 1 7-31 - it is noted that a timing controller exists in 
the user device for the subtitles to be synced to the proper presentation and displayed 
at the appropriate time denoted by the display time codes). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the user device of Zwaneveld to be a "portable" device to 
work in tandem with the caption storage, as taught by Hale, for the benefit of combating 
the problems presented by "open captioning" and presenting captioning conveniently to 
a user (Hale, col. 1, lines 35-41; col. 2, lines 1-4). 

Miyaoku teaches a user device having a microphone configured to receive a wireless 
acoustic signal that is time synchronized with the presentation for use in defining the 
timing during the presentation at which each caption is to be output to the user 
associated with the user device (Fig. 38; ffll [0205] - [0212], [0216] - [0224]; 
microphone receives audio from the broadcast and utilizes it to obtain more information 
on the broadcast video). 

Therefore, it would have been an obvious design choice to one of ordinary skill in the art 
at the time the invention was made to modify the portable device of Hale by fitting it with 
a microphone to receive acoustic voice signals which in turn can be used to garner 
more information on the video (like subtitles or closed captioning), as taught by 
Miyaoku, for the benefit of rendering to the viewer related information on the broadcast 
via a voice signal. 

Regarding claim 4, Hale further teaches the system according to claim 1 , wherein 
said captions include text and wherein said caption output circuit is configured to output 
said captions to a display device associated with the user device for display to the user 
(col. 3, lines 34-40; col. 5, lines 17-31 ; for closed captions include text and captions are 
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displayed on the screen display of the portable device 14). 

Regarding claim 5, Hale further teaches the system according to claim 4, wherein 
said captions include formatting information for controlling the format of the text 
displayed on said display (for the subtitles have certain formats which can be altered. It 
is noted that displayable texts are encoded with "formatting information" or instructions 
on how and where to appear on a display screen). 

Regarding claim 6, Zwaneveld teaches the system according to claim 4, wherein 
each caption includes duration information defining the duration that the caption should 
be displayed to the user (page 1 1 , lines 1-6; for the display cue stamp comprises 
display time stamp which denotes when the subtitles are to be shown. The display time 
stamp further denotes a beginning and the end of the display of a subtitle - reading on 
claimed duration of caption display). 

Regarding 7, Zwaneveld teaches the system according to claim 4, wherein 
said caption includes timing information defining the time at which the caption should be 
displayed to the user during the presentation (page 1 1 , lines 1 -6; for display time cues 
include display time marks denoting the timing information - beginning and end - of the 
subtitles). 

Regarding claims 9 and 10, Zwaneveld teaches the system according to claim 1 , 
wherein said presentation: includes audio and includes video (page 25, lines 6-1 0+; 
page 30, lines 6-1 3. See also Hale, col. 3, lines 34-40, 48-55 - for teaching on audio 
and video presentations). 

Regarding claim 12, Zwaneveld teaches the system according to claim 1, 
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wherein said caption store is formed in a memory card which is insertable into said user 
device and wherein said user device includes a reader for reading captions from said 
memory card when inserted therein (page 17, lines 6-12; page 29, lines 11-17; for the 
subtitle text files are stored in a memory medium - RAM, CD-Rom, floppy disk - which 
are portable; the caption storage element 1 1 can be embodied in a portable memory 
medium and sent to the user device - processing equipment - which would 
read/process it and send it to a display element. See also Hale, for teaching on 
storing of captions in the portable device and on memory cards - col. 2, line 63 - 
col. 3, line 3). 

Regarding claims 13-15, note the discussion on claim 12. Hale further teaches 
the system according to claim 1, wherein said caption store is provided in a computer 
system and wherein said user device includes a communication module for 
communicating with said computer system; wherein said computer system is remote 
from said user device; and wherein said user device includes a housing and wherein 
said communication module is provided within said housing (col. 2, line 63 - col. 3, line 
1 7; for it is noted that a docking station is used for connecting the portable device to a 
computer system; the docking station also may be used to connect the portable device 
to remote content sources. It is also well known that a mobile phone or PDA includes a 
housing and it is obvious that a communication module is housed inside to enable 
communication with a docking station). 
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Regarding claims 16, Hale further teaches the system according to claim 13, 
wherein said communication module is configured to communicate with said remote 
computer system using a wireless communication link (col. 3, lines 12-16). 

Regarding claim 17, Hale further teaches the system according to claim 16, 
wherein said user device comprises a mobile telephone or a personal digital assistant 
(col. 6, lines 43-46; PDA). 

Regarding claim 19, the combined teachings of Zwaneveld and Hale teach the 
system according to claim 42, wherein said synchronization information defines 
expected time points for one or more predetermined portions of the presentation (for the 
display time cues and time stamp indicates points denoting the portions of the 
presentation to output subtitle information. See also Hale for teaching on 
synchronization code associated with a beginning portion of a presentation). 

Regarding claim 20, Zwaneveld further teaches the system according to claim 
19, wherein said user device comprises a monitoring circuit configured to monitor said 
presentation to identify the actual time points of said one or more predetermined 
portions {for a "predetermined" portion of presentation can simply be the actual portions 
of the presentation that the user desires to receive closed caption or subtitle data; 
hence the actual time points of a predetermined portion is known by the user device 
because it is the device that user utilizes to request the closed caption or subtitle 
information. Hence giving this the broadest interpretation, the monitoring circuit in this 
case is merely checking or tracking the actual time in the presentation in which the user 
requested to receive caption information ) and wherein said timing controller is 
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responsive to the difference between the actual timings and the expected timings to 
control the outputting of the captions by said caption output circuit (for the display time 
stamps and cue identify the expected timing to output the caption information; the 
responsiveness of said timing controller to the difference between the actual timing and 
the expected timing is merely a form of error correction of synchronization signals. 
OFFICIAL NOTICE is hereby taken to the fact that error correction is well known 
concept in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to have made the timing controller responsive to 
the difference between the actual and expected times, as an element of error correction 
to ensure the correct captions are being outputted ). 

Regarding claim 21 , Miyaoku further teaches the system according to claim 20, 
wherein said predetermined portions of said presentation correspond to portions of 
audio of the presentation and wherein said monitoring circuit is coupled to said 
microphone for sensing the audio of the presentation and a includes comparator for 
comparing the received audio with the expected portions of the audio defined by said 
synchronization information flffl [021 6] - [0223]; for the received audio is processed and 
compared with the image-frame to check for correlation). 

Regarding claim 22, Zwaneveld further teaches the system according to claim 
20, wherein said user device has an acquisition mode of operation in which an output of 
said monitoring circuit is compared with said predetermined points defined by said 
synchronization information to identify a current position within said presentation and a 
tracking mode of operation in which the output of said monitoring circuit is compared 
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with a current predetermined portion defined by said synchronization information (page 
30, lines 1 3-22; for the output of the monitoring circuit is merely the time the user 
requests sets course to request for a subtitle, which is compared to the syncing display 
time stamp. Hence, the acquisition and tracking modes of operation are merely met by 
the comparison of audio signal signatures and associated subtitles reputedly each time 
the user requests subtitle information by 'collecting' an audio sample of the desired 
portion and sending it to the processor element 14). 

Regarding claim 23, see the discussion on claim 22. 

Regarding claim 24, the combined teachings of Zwaneveld, Hale and Miyaoku 
further teach the system according to claim 1 , wherein said user device is configured to 
receive said synchronization information from said caption store (Fig. 3; page 12, line 25 

- page 1 3, line 3; page 1 7, lines 6-1 2; for the sync info or time cue stamps together with 
the subtitle info are stored in the storage device 1 1, which forwards them to the 
processor equipment 14 or Hale's portable user device 14). 

Regarding claim 25, Zwaneveld further teaches the system according to claim 
1, wherein said synchronization information is embedded within said presentation and 
wherein said user device includes a monitoring circuit configured to monitor the 
presentation and to extract said synchronization information therefrom (page 12, line 25 

- page 13, line 3; page 30, lines 10-25). See also Miyaoku Iffl [0216] - [0224] for 
synchronization of acoustic signals with video presentation. 

Regarding claim 26, Miyaoku further teaches the system according to claim 25, 
wherein said synchronization information is embedded within the audio of said 
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presentation and wherein the monitoring circuit is responsive to the microphone to 
monitor the audio of the presentation (fflf [0216] - [0224]; obviously the monitoring 
circuit inside the user's terminal functions from the input of audio from the microphone). 

Regarding claim 27, Zwaneveld further teaches the system according to claim 
25, wherein said synchronization information comprises synchronization codes 
occurring at different timings during the presentation (page 12, line 25 - page 13, line 3; 
for synchronization codes are inherently embedded in the audio of the presentation to 
produce the synchronization information). 

Regarding claim 28, Zwaneveld further teaches the system according to claim 
27, wherein each synchronization code is unique to uniquely define the position in the 
presentation (for different or multiple display timings, it is noted that the synchronization 
information would also be different accordingly). 

Regarding claim 29, Zwaneveld teaches the system according to claim 1, 
wherein said caption store includes a plurality of sets of captions for a plurality of 
different presentations (for it is noted that the system provides subtitle information for 
different movies and films). 

Regarding claim 30, the combined teachings of Zwaneveld, Hale and Miyaoku 
further teach the system according to claim 29, wherein said user device is configured 
to capture a portion of said presentation and is configured to transmit the captured 
portion to said caption store and when said caption store is configured to use said 
captured portion of the presentation to identify the presentation being made and to 
transmit the associated set of captions for the identified presentation to said user device 
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(page 30, lines 4-26; for it is also obvious to modify the user device of Hale to capture 
the a part presentation and decipher the presentation being made and output the 
captions already stored in the user device, col. 5, lines 24-28). 

Regarding claim 31, note the discussion on claim 30. The combined teachings 
of Zwaneveld, Hale and Miyaoku teach the system according to claim 30, wherein said 
user device is configured to process the captured portion of the presentation to extract 
data characteristic of the captured portion and is configured to transmit said 
characteristic data to said caption store, and wherein said caption store is configured to 
use said characteristic data to identify the presentation being made and to transmit the 
associated set of captions for the identified presentation to the user device (page 30, 
lines 4-26; for the extracted characteristic data is audio signature of the captured portion 
of the presentation). 

Claim 33 is analyzed as the portable user device discussed in claim 1 . 

Regarding claim 36, the combined teachings of Zwaneveld, Hale and Miyaoku 
further teach a computer readable medium storing computer executable instructions for 
causing a general purpose computing device to operate as the user device of claim 1 
(page 30, lines 10-13; See also Hale, col. 2, lines 46-47 - memory and processors. See 
also Miyaoku, 1ffl [0303] - [0305]). 

Regarding claim 39, see claim 1 . The display cue time marks indicates the timing 
for the presentation of subtitle data and audio synchronization information synchronizes 
the audio with the proper subtitle (page 12, line 25- page 13, line 3). 

Regarding claim 40, see claim 1 . 
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Regarding claim 42, Hale further teaches a system according to claim 1, 
configured to receive synchronization information defining the timing during the 
presentation at which each caption is to be output to the user associated with the user 
device and wherein said timing controller is configured to use the received wireless 
[acoustic] signal and the received synchronization information in determining the timing 
during the presentation at which each caption should be output to the user (col. 4, line 
61 - col. 5, line 1 5 - for caption text content is synchronized with a presentation and 
displayed by the portable device, the synchronization made possible by time code 
signals received wirelessly from a transmitter, wherein the portable device plays the 
caption texts at appropriate time synchronous to the presentation). 
Miyaoku further teaches receiving wireless acoustic signal and a synchronization 
information flffl [0205] - [0212], [0216] - [0224]). 

Regarding claim 43, Hale further teaches a system according to claim 1, wherein 
the first receiver is configured to receive a set of captions or said sequence of captions 
via a telephone network (col. 3, 4-7; col. 6, lines 43-47 - Hale, although not explicitly 
stating it, fairly suggests that the portable device can be a mobile telephone ~ "similar 
handheld with wireless communications receiver." An ordinary skilled artisan would 
understand and appreciate that a mobile telephone can be a portable device intended 
for use with Hale's invention; and it is obvious for a mobile telephone to communicate 
over a telephone network - see also, col. 3, lines 14-17). 

Regarding claim 44, Hale further teaches a system according to claim 1 , wherein 
said first receiver is configured to receive said set of captions or said sequence of 
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captions over a wired communications link in advance of the presentation (col. 3, lines 
4-7, 17-21). 

Regarding claim 45, Hale further teaches a system according to claim 1, wherein 
said user device is configured to use said first receiver to download a next caption from 
said caption store when it detects a synchronization code in the wireless signal received 
from said microphone (of Miyaoku) (col. 3, lines 22-33 - for use in downloading the 
caption content from a remote server by the portable device only when they are needed 
and thereby requested). 

Regarding claim 48, note the discussion on claim 43. Hale further teaches a 
system according to claim 1 , wherein said caption store is provided by a remote server, 
wherein said user device comprises a mobile telephone and wherein the user device is 
configured to allow said user to interact with the remote server using the mobile 
telephone (col. 3, lines 10-16). 

Claim 49 is analyzed as a device of claims 4 and 5. 

Claims 50 and 51 are analyzed as devices of claims 6 and 7 respectively. 

Claim 52 is analyzed as a device of claims 13 and 14. 

Claims 53 and 54 are analyzed as devices of claims 16 and 17 respectively. 

Claim 55 is analyzed as a device of claim 42. 

Claims 56-60 are analyzed as devices of claims 19-23 respectively. 

Claim 61 is analyzed as a device of claim 25. 

Claims 62-65 are analyzed as devices of claims 27, 28, 30 and 31 respectively. 
Claim 66 is analyzed as a method of claim 1 . 
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6. Claim 32 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zwaneveld in view of Hale in view of Miyaoku and further in view of Ogasawara 
(US 6,512,919). 

Regarding claim 32, the combined teachings of Zwaneveld, Hale and Miyaoku 
teach the system according to claim 1 , wherein said presentation is given at a venue 
(for a movie or film theatre is a venue. Further, venue broadly reads on any location). 
However, they do not explicitly teach wherein said venue is configured to provide an 
activation code, wherein said user device is configured to receive said activation code 
and further comprises a inhibitor for inhibiting the operation of said caption output circuit 
unless said user device has received said activation code. 

Ogasawara teaches a venue configured to provide an activation code, wherein 
said user device is configured to receive said activation code (col. 3, lines 6-17; col. 9, 
lines 6-14; 33-42; for the purchase transaction program is downloaded from the server 
to the mobile telephone, thereby reading on an activation code, because without the 
transaction program the videophone cannot function as a personal electronic shopping 
system) and further comprises a inhibitor for inhibiting the operation of said caption 
output circuit unless said user device has received said activation code (for without the 
purchase transaction program being sent to it, the phone can't function as a shopping 
system; hence it is obvious that the phone has an 'inhibitor' or access denial 
mechanism). 

Therefore, it would have been obvious to an ordinary skilled artisan to modify the 
portable user device to include an access mechanism by receiving a program or access 
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code to access the caption store, as taught by Ogasawara, for the benefit of ensuring a 
privacy and proper identification of the user. 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dika C. Okeke whose telephone number is (571)270- 
5367. The examiner can normally be reached on Monday - Thursday, 9:00 a.m. to 7:00 
p.m., EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian T. Pendleton can be reached on (571)272-7527. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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